1. Field of the Invention
The present invention relates to an augmented reality apparatus and method and, more particularly, to an augmented reality apparatus and method in which the direction of a camera is adjusted to be consistent with a direction toward which a user faces and an augmented reality image is realized for a subject image incident from the adjusted camera direction.
2. Description of the Related Art
Recent developments in integration technology have opened the way for the spread of various portable devices such as audio reproduction apparatuses (e.g., cassette tape players and CD players), wireless pagers, mobile phones, MP3 players, and portable game players.
In particular, mobile phones are no longer only tools for telecommunications but are also tools for providing a variety of functions such as a clock function, a digital camera function, and an MP3 player function.
Virtual reality is one of the relatively new information technology paradigms that can allow people to experience various situations that they will never experience in real life due to some spatial and physical restrictions, by creating a computer-based artificial environment that interactively responds to the human sensory system. Virtual reality aims to establish a more immersive environment where humans and computers can effectively interact and communicate by enhancing human-computer conversation skills using a variety of input/output techniques. Such input/output techniques involve the use of, for example, head-mounted displays (HMD), data gloves, or motion capture systems. HMDs obtain data regarding variations in the position of a user by monitoring the movement of the head of the user, and transmit the data to a computer, thereby allowing the computer to calculate through simulation the size and depth of an object located within the viewing angle of the user.
Data gloves detect the positions and directions of the hands of a user who wear them. When the user moves his/her hands in a three-dimensional (3D) space, the data gloves transmit 3D coordinate data representing the movement of the hands of the user to a computer. Then, the computer manipulates an object corresponding to the data transmitted by the data gloves.
Augmented reality, which is the term derived from terminology regarding virtual environment and virtual reality, is a field of computer research which deals with the combination of real-world images and virtual-world images such as computer graphic images. Real world information may include information that is not necessarily needed by users or lack information that is needed by users. In a virtual environment created by a computer, such unnecessary information can be simplified or can be made invisible. In other words, augmented reality systems combine a real environment with virtual objects, thereby effectively interacting with users in real time.
Augmented reality has been widely used in various fields of application such as the entertainment field and the newscast field. We often see TV weather broadcasts where the forecaster appears in front of a weather chart that keeps changing naturally. In reality, the forecaster stands in front of a blue screen, and a virtual studio environment is established based on computer-generated virtual images, thereby realizing augmented reality.
Augmented reality can also be applied to the medical field. In detail, 3D data regarding a patient is obtained in real time using a medical imaging technique such as magnetic resonance imaging, computed tomography imaging, and ultrasound imaging, and the obtained 3D data is rendered as virtual images. Then, the virtual images are displayed by laying them over the patient, thereby enabling and facilitating image-guided medical surgeries. In addition, augmented reality can also be applied to the visors of the helmets of pilots or the windshields of cockpits to display virtual images and thus to present a variety of information necessary to the pilots.
In order to realize augmented reality, a camera which can receive real images and a display device such as an HMD which can display both real images and virtual images are needed. However, existing augmented reality systems are very expensive, and thus, are not affordable to most people.
In the meantime, Japanese Patent Laid-Open Gazette No. 2004-038470 discloses a mixed reality apparatus and method in which a real image is captured using an imaging unit, the direction toward which a user faces is determined based on the captured real image by a position/direction determination unit, the relationship between the position of the user and the position of a predetermined object is calculated, and data obtained as the result of the calculation is displayed in a virtual space in such a manner that the data can be laid over the captured real world image.
This mixed reality technique, however, involves detecting the direction toward which a user who wears an HMD faces and displaying an image incident from the detected direction, and thus needs additional equipment such as an HMD, like other existing augmented reality techniques. Therefore, it is necessary to develop methods to display augmented reality images according to the viewing angle of a user by using, for example, mobile phones which are relatively widespread.